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Implications of difficulties in using intuitive 
statistical reasoning are considered as they bear on the theoretic 
model viewing persons as intuitive scientists who make causal 
attributions in everyday lite. Data sampling strategies and use of 
judgment rules to identify covariant events were investigated in two 
studies. Using a rule-analytic method, Study 1 probed the 
relationship between rules of causal and covariation judgment among 
junior high and college students. Findings indicated that the faulty 
rules of covariation judgment so common among the subjects resulted 
in incorrect assessments of many covariation relationships. Further 
analyses revealed that males and females produced notably different 
strategy distributions in response to the causal and covariation 
questions. Study 2 investigated information sampling strategy as a 
potential constraint on the adequacy of everyday causal reasoning 
among third grade, seventh grade, and college students. Aa in Study 
1, subjects frequently failed to identify true event covariates as 
causes, often accepting noncovar iates as causes of events. Although 
causal judgment accuracy increased with age, errors were coranon even 
among college subjects, especially when judging noncontingent 
relationships. Information sampling strategy was found to be one 
potential contributor to relative accuracy in causal judgment. 
Generally, individuals' statistical intuitions were too faulty to 
support accurate causal inference. (RH) 
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In the 1 9 6 0 ' s and 7 0 ' s , theories of causal reasoning were 
dominated by an iraage of people as intuitive scientists. 
Fr eminent theories within this tradition included attribution 
theory in social psychology, and Piaget's model of formal 
operational reasoning. In these theories, people were said to 
draw conclusions about real world causal relationships by 
identifying several possible causes of the event to be explained, 
and collecting evidence relevant to those possible cause:. Cr.ce 
the information was gathered, the intuitive scierviist would 
evaluate the relative contributioriS of the alternative causes i:c 
determine the best account of the event. 

However, the real world is a difficult content in which to 
test causal hypotheses, filled as it is with less-thar. - perfect 
relationships between events. Considering the many exceptions 
the intuitive scientist might find to any causal rule, how was he 
or she to identify reliable relationships? By the use of 
intuitive statistics, according to the model. In particular, 

people would sample data about the co-occurrence of an event and 
its possible causes in order to assess the e:;tent to which the 
two events covary. Since causes and their effects do covary, 
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this process would effectively define the set of possible causes 
of any event. If covariation judgment plays the central role in 
causal reasoning suggested by this model, then the quality of the 
intuitive scientist's causal judgments would depend on the 
adequacy of his or her statistical intuitions. Two aspects of 
statistical reasoning are likely to be of special importance: 
data sampling and judgment rule. In fact, existing research 
indicates that laypeoples' intuitions about both of these aspects 
are likely to be sources of error in causal judgment. 

Studies of information sampling in hypothesis testing commonly 
show that people select only c subset of the potentially relevant 
information in testing hypotheses. In the case of causcl 
judgment, a subject asked whether an outcome is caused by a 
particular event might prefer to sample information about the 
outcome \;hen that possible cause is present, gathering less 
information about event frequencies when the possible cause is 
absent. A biased sample such as this may include too fe*.: cases 
of the minority event to provide a reliable i n c e :; of the 
proportions in the population. The problem would be most severe 
in the case of the most extreme bias in sampling. That is, if 
an individual only gathers data under one event state and never 
samples information about the alternative state, judgment cannot 
be at better than a chance level cf accuracy. Such an 
individual may reliably assess the likelihood of the outcome when 
the supposed cause is present, but would not knew if that 
likelihood is any different when the cause is absent. In this 
way sampling strategies may contribute to relative causal 
judgment accuracy. 
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Once information has been sampled, the intuitive scientist 
must decide if the two events are, in fact, related, However, 
investigations in a variety of laboratories converge in 
identifying a panoply of problems that people have in judging 
such event covariations, Host relevant to the present concern 
are studies cf the rules people use to identify event 
covariations. De v e 1 opmen t a 1 studies have shown that children 
begin to use systematic but overly simple rules of covariation 
j udgment by the early elementary school years (Shaklee and I! ins , 
1231; Shaklee and Paszek, 1 9 8 5). II ore complex rules become 
common with increasing age but, even in college populations, only 
a minority of subjects show a mathematically accurate 
understanding of covariation (Shaklee and ilims, 19G1, 1932; 
Shaklee and Tucker, 19G0; Shaklee and Hall, 1903). In fact 
about <: third of adult subjects in these studies were classified 
as using the same simple and eiror-prone r.ule that is sc 
prominent among second grade children. 

In this presentation, I would like to consider the 
implications of people's problems with statistical reasoning for 
the model of people as intuitive scientists. In particular, we 
will report evidence about how older children and adults collect 
and evaluate data in order to reach a conclusion about cause- 
effect relationships. Two aspects of the process will be of 
central interest here: information s amp ling and j udgmen t rule. 

Our first study investigates the implications of peoples' 
rules of covariation judgment for their inferences about causal 
relationships. Four rules have beer, commonly proposed as bases 



of covariation judgment. Two of the least sophisticated of the 
proposed judgment rules would draw a conclusion about event 
covariations without using all of the information in a two-by-two 
contingency table* By the cell-* rule, judgment if, mace 
according to the frequency with which the target events co-occur. 
For example, such a rule would identify a positive relationship 
between plant food and plant health if there are core healthy 
plants with plant food than any of the other event-state 
combination So A second simple approach would compare the 
frequency of target event states with and without the supposed 
covariate (e.g. healthy plants with plant food vs. healthy plants 
without plant food, comparison of cc r.t ing e::cy table cells a and 
b , strategy a-versus-b). By a much improved approach, an 
individual would compare the number of event-state combinations 
confirming a positive relationship between plant food aud plant 
health, i.e. healthy plants with plant food and unhealthy plants 
without plant food, with the number of event-state combinations 
disccnf irming the relationship, i.e. healthy plants without plant 
food and unhealthy plants with plant food. Since this strategy 
compares the sums of diagonal contingency table cells a+d and 
b+c, this rule is referred to as the sum of diagonals rule. 
Finally, an individual might determine the relationship by a: 
optimal rule: comparing the likelihood of the target event-ctate 
under each of the two conditions (e.g. P( healthy plants/plant 
food) vs. P( healthy plant/no plant food)), conditional 
probability rule. This is the only one of f.he four rules v;hich 
would accurately judge all interevent contingencies. 

This analysis of possible rul has allowed us to 
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discr iciinate arsons strategies actually euployed by subjects of 
various ages in waking covariation judgments. That is, 
different rules should produce different judgments on carefully 
selected covariation problems. A set of such problems is 
illustrated in Table 1. Solution accuracy is indexed by the 
direction of the judged relationship. Problems are structured 
hierarchically such that cell-a problems are correctly solved by 
all strategies, a-versus-b problems are accurately solved by all 
strategies e::cept cell-a, sua of diagonals problems are 
accurately judged by sun-of- diagonals and conditional 
probability strategies, and conditional probability problems are 
accurately judged by the conditional probability rule alor.c (sea 
Table lb). 

Fast arguments of a covariation-causal judgment link v/ould 
predict that subjects should prefer as causes those events v;hich 
they identify as event covariates. The strategies subjects use 
to judge covariations between events should be the sane as those 
employee to define cause-effect relationships. The present 
investigation uses our rule analytic net hod to identify the 
relationship between rules of causal and covariation judgment. 

Junior high (grades 6, 7, S) and college subjects were asl;ed 
to m a 1; e either causal or covariation judgments of a set of 12 
different problems, specially structured to yield a distinctive 
pattern of solution accuracy by each of the four proposed 
judgment rules. Each problem v;as sot in a concrete conte:;t of 
t v c everyday events t: h i c 1: cay or may net be related. Subjects 
;:;-r: shorn iuctance: cf c v e r. t - s t a t 2 c o:- It i:*.a t ion s organized i:. a 
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211 2 table and were asked to make either a c:uscl or covcrict ior 
judgment about the relationship shown. In the case of the 
covariation condition subjects were asked: 

The picture shows that plants were nore likely to be healthy if 

a) it was foggy. 

b) it was not fog^y. 

c ) no difference. 

The causal question for the same problem was: 
The picture shows that plant food: 

a) r.£/.2S plants healthy. 

b) keeps plar.zs fron being healthy. 

c) has no effect on plants' health. 

Subjects' judgments in this experiment inform us in two ways 
about the link between event covariation and causal judgment. 
First, did subjects consistently select true event covariates as 
causes? As we night have expected from our prior research in 
covariation judgment, the answer to this question is no. In the 
present set of problems, junior high subjects identified 
covariates as causes only 49£ of the tine, college subjects oi: 
S5£ of the problems. The data in Table 2 indicate the percentage 
of problems of each type in which subjects judged the causal 
relationship to be in the sane direction as the depicted 
covariation. As the table shows, subjects did consistent ly judge 
covariates as causes for the cell-a and a-versus-b problems. 
The strong influence of problem type points to the strategic 
basis of these judgment patterns. That is, the faulty rules of 
covariation judgment so common among these subjects result in 
incorrect assessments of many covariation relationships. 



However, we can use the data to ask a second question about 
the supposed c a u s e -c o v a r i a t e link. That is, are subjects' 
causal judgments based on their intuitive notions about 
covariation? By this interpretation, subjects' rules of 

covariation judgment (however faulty) should natch their rules of 
causal reasoning. Table 3 shows the rule classifications of 
junior high and college males and females for causal and 
covariation judgments. 

Classifications of subjects ir. the two grades snow parwllsl 
age trends for covariation and causal judgment, in support cr 
this interpretation. The present data indicate that 

improvements in ccvariation judgment during these years are 
matched by progress in causal reasoning as well. These data fit 
veil with the interpretation that subjects look for covariates as 
causes throughout these years. However, ioproved rules of 
covariation judgment enable older subjects to more accurately 
find true covariates when explaining events. 

However, the interpretation further requires chat subjects 
use the sane rules to cake causal and covariation judgments. 
Hale subjects' judgments fit well with this prediction: 
distributions of strategies for covariation and causal judgments 
are quite comparable for male subjects in both the junior high 
and college samples. Females, on the other hand, produce 
notably different strategy distributions in response to the 
causal and covariation questions. As shown in the table, in the 
junior high sample, females' classifications indicate that they 
rely heavily on the very simple cell-a strategy for covariation 
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judgment. However, when making causal judgments they wore more 
likely tc use the a-versus-b and sun of diagonals strategies. 
The increase in un c 1 a s s i f iab 1 e strategies among both nales and 
females among these younger subjects indicates that several of 
the subjects failed tc use any of our proposed strategies, 
perhaps using some other strategy or no consistent strategy at 
all. Inspection of these unclassified judgment records failed 
to reveal any systematic bases of these judgment patterns. 

College females also show use of different strategies for the 
causal and covariation judgments, using mere co::?le:: judgment 
s : . : t G g i e s when asked about causes than when asked about 
covariations. These subjects were most likely to use the mGrc 
accurate sum of diagonals and conditional probability rule! in 
response to the causal question. In contrast, responses to the 
covariation question show a strong consensus toward use of the a- 
versus-b rule. This is ironic in view of the fact that the 
covariation question directly asks for a comparison of 
conditional probabilities, i.e. is a given outcome more likely 
given condition Bl, or condition 32. Hule classifications 
indicate that, overwhelmingly, college women respond to this 
c.uection by comparing frequencies, not probabilities. 

Thus, the outcome of this study offers qualified support for 
the much-suggested link between covariation and causal judgment. 
People in the junior high to college age span are net 
particularly good at identifying true covariates as causes of 
events. Kovever, parallel rule use when judging causes and 
covariates suggests that males, at least, may define causes and 
covariates in the same terms. Females, in contrast, seem to 



define the two concepts by different rules « 

How ever, our model of the intuitive scientist suggests yet 
another major aspect of causal judgment in which intuitive 
statistics may play a role: i.e. information sampling. Our r.e;;t 
study investigates information sampling strategy as a potential 
constraint on the adequacy of everyday causal reasoning. 

In this study, subjects in third grade, seventh grade and 
college were ashed to test hypotheses about causal relationships 
between the state of a plant (e.g. flowers open or closed) and 
the lighting conditions under which it grew. For each problem, 
subjects were given two envelopes, one containing observations of 
plants growing in the sun, the other containing observations of 
plants growing in the shade. They were to select fro:: these 

envelopes a total of 24 observations of the plant growing in the 
sun anc/or in the shade. Subjects recorded their observations, 
l.ien judged whether the stated hypothesis was true. Hypotheses 
were stated either in terms of the effects of the sun (e.g. Sun 
makes spots on leaves) or the effects of the shade (e.g. Shade 
makes spots on leaves). Subjects judged three such problems, 
including one none on t ingen t relationship in which the plant was 
in a given state in 7 5 % of the observations in both the sur. and 
the shade, and two contingent relationships (plant in a given 
state in 75% of the sun observations, 25% of the shade 
observations). In one contingent problem, the direction of the 
contingency matched the causal hypothesis so that the correct 
anrswer to the causal question would be yes (contingent-yes). In 
the other problem, the contingency was opposite in direction to 
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the stated hypothesis, so that an accurate judge would reject the 
causal hypothesis (contingent-no). 

First we can ask how accurate subjects were at judging these 
causal relationships. A judgment was scored as accurate if the 
subject judge, the causal relationship as being in the Saae 
direction as the covariation relationship. Accuracy could be 
evaluated either in comparison to the proportions of event-states 
represented in the envelopes fro. which the subjects sampled 
(population-based accuracy), or in relation to the samples they 
actually drew ( s amp 1 e -b a s e d accuracy). These tw0 differen ^ 
definitions of accuracy produced essentially the sa,e results, so 
we will restrict our discussion to the results of the population- 
based measure. 

These analyses show that accuracy of causal judgments 

improved with age, with means represented in Table 4, and that 

the noncontingent problem was harder than the two contingent 

?roble,:s in all age groups. Thus, as in our previous 

experiment, subjects frequently failed to identify true event 

covariates as causes, often accepting non c o v ar i a t e s as causes of 

events. Although causal judgnent accuracy increases with age, 

errors were common even anong college suhieet* . an .-4..ii 

o *.wij.c 0 e &uDjeccs, especially v:nen 

judging noncontingent relationships. 

Information sampling strategy is one potential contributor 
to relative accuracy in causal judgment. Subjects did 
frequently draw biased samples, with the extent of that bias 
increasing somewhat with age (mean absolute difference of sun c,cl 
shade observations = 8.7, 6.2, 5.9 for grades 3, 7 and college 

students respectively). c.-.i ,•„, u • , , . 

^ t4VB *>'« bcmplmg bias showed its greatest 
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cffact o:i causal judruent accuracy in interaction with iihe form 
of the causal question. In each of the three problems, subjects 
who drew biased samples of the sun and shade observations were 
:-'ibs tant ial ly different in accuracy depending on whether they 
were asked whether sun or shade caused a given outcome. As 
shown in Table 5, subjects showed near-chance accuracy when asked 
about the effect of the event about which they had sampled few- 
cases. However, when asked about the majoi-ity event, subjects' 
responses were consistent with the proportions represented within 
t h a t e v e t . 

These accuracy patterns nay be interpreted in terns of the 
subjects' rules for combining the frequency information into a 
causal judgment. The appropriate approach to the causal 

question would be to compare rates of the target outcome (e.g. 
open flowers) in the two growing conditions (sun and shade). 
Someone who uses this rule should be similarly accurate whether 
the question is phrased in terns of the effect of the sun or the 
shade. The differential accuracy observed here indicates that 
subjects used some other rule for causal judgment. 

Ue suggest that subjects judged the causa?, relationship by 
looking at the proportions of event states in the condition about 
which they were asked (e.g. Does sun make?...), ignoring the 
equally relevant cases in the alternative condition. This would 
be a variant of the tendency to ignore base rates seen in a 
variety of contexts in judgment research (Arkes and P.othbart, 
1935; 3ar-Hillel, i980; Beyth-Maron & Fischhoff, 1933). That 
is, subjects consider the ratio of event-s ,tes in the presence 
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of the supposed cause, without regard for the event-state base 
rates when the cause is absent. 

For the two contingent problems, this strategy could yield 
the correct answer. For ex an pie, in one problem, the ratio of 
open to closed flowers was 3:1 in the sun and 1:3 in the shade. 
I' alf of the subjects were asked if sun caused the flowers to 
open. Subjects who s amp led heavily f r on the sun envelope were 
very accurate in judging this relationship ( S 9 Z ) ; subjects who 
primarily sampled observations in the shade were at chance level 
accuracy (54%). Other subjects saw the same ratios of open accl 
closed flowers and were asked about the effects of shade 
(contingent-no, shade). These subjects were more accurate if 
they had sampled primarily shade observations (G2% vs 61 % 
correct), lie would suggest that, in all cases, subjects answered 
the question in terms of the outce::a frequencies in the condition 
about which they were ashed. In these problens,. the direction 
of differs::: target-state outcomes within that condition 

(e.g. more open than closed flowers within the sun) matched the 
true contingency in the problem. Those who had sampled 
dominantly in that condition had a reliable sample of evenc state 
outcomes within that condition, hence were accurate in judgment. 
Subjects who had sampled doninantly from the other condition had 
a poor sample on which to base their judgment, thus were less 
accurate. The contribution of sampling bias to this 

differential accuracy is further indicated by looking at 
subjects who sampled equally from the two conditions. For each 
problem, accuracy levels were more similar for the sun and shade 
question forms for these unbiased subjects than for the two 
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biased sample groups. 

This proposed strategy receives its clearest support in the 
noncont ingent problem where it will lead the subject to the wrong 
conclusion about the causal relationship no natter which sampling 
bias is used. Both lighting conditions in this problem show the 
sa:;e 3:1 ratio of open to closed f lovers. Subjects who s an pled 
primarily fron the cun envelope erroneously concluded that the 
sun caused the flowers to open (71%); subjects who sampled 
doninantly fron the shade decided that the shade made the flowers 
open (77%). IJote that these two groups reached opposite 
conclusions about the same relationship, and that both of those 
conclusions are wronj. Ue would suggest that subjects in each 
case vere impressed with the predominance of open flowers in the 
condition sampled and failed to consider the possibility that 
open flowers might occur at the same rate in the alternative 
condi t ion • 

In overview, the combined results of these experiments show 
the later childhood and adolescent years to be periods of active 
development in these more complex aspects of causal reasoning. 
Causal judgment accuracy improves over these years, but judgment 
errors continue to be common even at adulthood. The evidence 
further suggests that faulty strategies of information sampling 
and rules of covariation judgment undermine the abilities of 
subjects in these age ranges to make accurate causal inferences. 

In light of these findings, let's reconsider the model of 
people as intuitive scientists in everyday causal reasoning. If 
the model is interpreted to mean that people make accurate 
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statistically - based causal i i: f 2 r s n c 2 s , the present evidence 
raises serious questions about the model. People's statistical 
intuitions are just too faulty to support accurate causal 
inference. 

However, if the model simply implies that causal judgments 
are based on people's impressions about event covariations 
(faulty though they nay be), the present evidence offers souie 
support for the notion. In fact, these studies find a central 
role for statistical reasoning in processes of causal inference. 
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Judging Causes and Covariates: 
Strategies of Older Children and Adults 

Harriet Shaklee 
Eugene Research Institute, Eugene, OR 



Table 1 



A) Sample covariation problems 



Cell a 
Problem 



£ versus J) 
Problem 



Sum of Diagonal 
Problem 



Conditional 
Probability 
Problem 





B l 


B 2 




B l 


B 2 




B l 


B 2 




B l 


B 2 


A l 


2 


11 




4 


11 


A l 


8 


8 


Al 


12 


2 


A 2 


7 


4 


A 2 


8 


1 


A 2 


8 


0 


A 2 


10 


0 



B) Strategy use and resultant patterns of problem accuracy 
(+ * accurate, 0 » inaccurate) 

Problem Strategy Type 

Sum of Conditional 
Cell £ £ versus b Diagonals Probability 



Subject 

Strategy 

Type 



Conditional 
Probability 



Sum of 
Diagonals 



a versus b 



Cell a 



Paper presented to the Society for Research in Child Development 
Baltimore, MD, April 23 - 26, 1987. 
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Table 2 

Percent Correct per Problem Type 



Problem Types 





Cell 
a 


a_ versus b 


Sum of 
Diagonals 


Conditional 
Probability 


All 


Junior High 


96.3 


92.7 


.16.0 


8.0 


53.0 


Covariation 












College 


95.7 


98.7 


43.0 


33.7 


67.7 


Junior High 


75.7 


60.0 


31.0 


28.7 


48.7 


Cause 












College 


88.0 


80.0 


53.0 


38.3 


64.7 
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Table 3 

Strategy Classifications by Question and Gender at Each Age (Percentages) 

Strategy Classifications 

Unclas- Sum of Conditional 

Gender sified Cell-a a-vs-b Diagonals Probability N 

Junior High 

Covariate 

Male 0.0 10.0 70.0 20.0 0.0 10 

Female 14.3 78.6 7.1 0.0 0.0 14 

All 8.3 50.0 33.3 8.3 0.0 24 

Cause 

Male 33.3 0.0 44.4 11.1 11.1 9 

Female 38.5 23.1 23.1 15.4 0.0 13 

All 36.4 13.6 31.8 13.6 4.5 22 

College 
Covariate 



Male 


4.0 


0.0 


32.0 


20.0 


40.0 25 


Female 


0.0 


0.0 


86.2 


0.0 


13.8 29 


All 


1.8 


0.0 


61.1 


9.3 


27.8 54 








Cause 






Male 


0.0 


13.0 


30.4 


26.1 


30.4 23 


Female 


7.1 


3.6 


42.9 


25.0 


21.4 28 


All 


4.0 


7.8 


37.2 


25.5 


25.5 51 
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Table 4 

Accuracy of Causal Judgment: 
Fercent Correct by Problem and Subjects' Grade 



Grade 



Problem 


3 


7 


College 


All 




Population-based 


Accuracy 






Noncontingent 


44.0 


27.3 


60.0 


45.0 


Contingent-yes 


68.0 


81.8 


78.2 


75.8 


Contingent-no 


60.0 


79.5 


98.2 


79.9 


All 


57.3 


62.8 


78.8 


66.9 




Sample-based Accuracy 






Noncontingent 


55.8 


54.0 


66.7 


59.2 


Contingent-yes 


61.9 


78.9 


78.2 


73.0 


Cc.^tingent-no 


57.1 


75.7 


100.0 


78.6 


All 


58.3 


69.5 


81.6 


70.3 
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Table 5 

Population-Based Accuracy of Causal Judgment by Direction of 
Information Sampling Bias for Each Problem Contingency 
(Percent correct, N's in parentheses) 



Direction of Sampling Bias 

Problem 

Contingency Question Shade * Sun Shade > Sun Sun > Shade 



Noncontingent 


Sun 


55.9 


58.3 


29.0 






(34) 


(13) 


(31) 




Shade 


42.9 


23.5 


68.4 






(35) 


(17) 


(19) 


Contingent-yes 


Sun 


69.0 


53.8 
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Question : Form of the causal question: Does the sun (or shade) 
make flowers open? (or other target state) 
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